refactor: remove unshipped reflection service by yyiilluu · Pull Request #360 · ReflexioAI/reflexio

yyiilluu · 2026-07-18T06:38:42Z

Summary

Remove the unshipped, citation-driven ReflectionService in one clean change without deprecation shims.
Delete its public client/config surfaces while preserving the supported profile and playbook learning paths.
Ignore and remove the retired reflection_config key when loading legacy private local configuration.

Changes

Runtime and public API

Remove run_reflection, ReflectionConfig, reflection service wiring, deferred/durable orchestration hooks, and reflection billing/model registry entries.
Keep generation, profile extraction, playbook updates, and durable learning behavior on their supported paths.
Add durable publish-to-worker coverage for the remaining learning pipeline.

Prompts, evals, and documentation

Delete the reflection prompt family, service implementation, dedicated eval harness, fixtures, and tests.
Update scenario/consolidation evals and internal package documentation so they no longer depend on reflection.
Add regression coverage proving removed config/API symbols stay unavailable.

Test Plan

OSS unit/integration suite: 4722 passed, 73 skipped, 6 subtests passed
OSS E2E suite: 43 passed, 49 skipped
Focused removal and configuration suites: passed
Ruff lint/format and Pyright: passed
Removed-symbol repository audit: no remaining runtime/public references

Summary by CodeRabbit

New Features
- Durable learning now focuses on profile and playbook updates without a separate reflection stage.
- Legacy reflection configuration is removed from the public configuration model.
Bug Fixes
- Legacy configuration files are automatically cleaned up while preserving settings, even if cleanup is temporarily unsuccessful.
Documentation
- Updated service, attribution, lineage, and evaluation documentation to reflect the streamlined learning workflow.
Tests
- Added coverage for legacy configuration cleanup and profile/playbook persistence.

Delete the public API, config model, orchestration, prompts, evals, and service-only tests. Preserve legacy private config loading by stripping the retired raw key before strict validation.

coderabbitai · 2026-07-18T06:39:10Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 9b49f2b2-8eb4-49e2-95aa-2e475f7a92c0

📥 Commits

Reviewing files that changed from the base of the PR and between 76ec3ed and 3eba9d2.

📒 Files selected for processing (83)

reflexio/README.md
reflexio/lib/_profiles.py
reflexio/lib/_reflection.py
reflexio/lib/reflexio_lib.py
reflexio/models/api_schema/domain/entities.py
reflexio/models/config_schema.py
reflexio/server/README.md
reflexio/server/billing_meter.py
reflexio/server/prompt/prompt_bank/memory_reflection/v1.0.0.prompt.md
reflexio/server/prompt/prompt_bank/memory_reflection/v1.1.0.prompt.md
reflexio/server/prompt/prompt_bank/memory_reflection/v1.2.0.prompt.md
reflexio/server/prompt/prompt_bank/memory_reflection/v1.3.0.prompt.md
reflexio/server/prompt/prompt_bank/memory_reflection/v1.4.0.prompt.md
reflexio/server/prompt/prompt_bank/memory_reflection/v1.5.0.prompt.md
reflexio/server/prompt/prompt_bank/memory_reflection/v1.6.0.prompt.md
reflexio/server/prompt/prompt_bank/memory_reflection/v1.7.0.prompt.md
reflexio/server/services/README.md
reflexio/server/services/configurator/local_file_config_storage.py
reflexio/server/services/configurator/test_config_storage.py
reflexio/server/services/deferred_learning_plan.py
reflexio/server/services/generation_service.py
reflexio/server/services/playbook/playbook_edit_apply.py
reflexio/server/services/reflection/__init__.py
reflexio/server/services/reflection/components/__init__.py
reflexio/server/services/reflection/components/extractor.py
reflexio/server/services/reflection/reflection_service_utils.py
reflexio/server/services/reflection/service.py
reflexio/server/services/storage/storage_base/playbook/_user.py
reflexio/server/services/storage/storage_base/profiles/_profile_store.py
reflexio/test_support/llm_model_registry.py
tests/e2e_tests/test_contradiction_resolution_e2e.py
tests/e2e_tests/test_failure_path_polarity_e2e.py
tests/eval/consolidation/README.md
tests/eval/consolidation/case.py
tests/eval/consolidation/judge.py
tests/eval/consolidation/providers.py
tests/eval/extraction/README.md
tests/eval/extraction/providers.py
tests/eval/extraction/runner.py
tests/eval/prompt_deviation_guard.py
tests/eval/reflection/README.md
tests/eval/reflection/__init__.py
tests/eval/reflection/case.py
tests/eval/reflection/fixtures/__init__.py
tests/eval/reflection/fixtures/illustrative_cases.json
tests/eval/reflection/judge.py
tests/eval/reflection/providers.py
tests/eval/reflection/runner.py
tests/eval/reflection/test_reflection_eval.py
tests/eval/scenarios/README.md
tests/eval/scenarios/book.py
tests/eval/scenarios/case.py
tests/eval/scenarios/fixtures/scenarios.json
tests/eval/scenarios/runner.py
tests/eval/scenarios/test_book.py
tests/eval/scenarios/test_scenario_eval.py
tests/eval/test_prompt_deviation_guard.py
tests/models/test_reflection_config.py
tests/models/test_removed_reflection_config.py
tests/server/llm/test_litellm_client_unit.py
tests/server/services/durable_learning/test_compute_persist_split.py
tests/server/services/durable_learning/test_worker.py
tests/server/services/playbook/test_consolidation_lineage_integration.py
tests/server/services/playbook/test_playbook_edit_apply.py
tests/server/services/playbook/test_playbook_generation_service.py
tests/server/services/prompt/test_dispatchers.py
tests/server/services/reflection/__init__.py
tests/server/services/reflection/test_generation_service_wiring.py
tests/server/services/reflection/test_module_contract.py
tests/server/services/reflection/test_post_horizon_filter.py
tests/server/services/reflection/test_reflection_profile_supersede_integration.py
tests/server/services/reflection/test_reflection_service.py
tests/server/services/reflection/test_reflection_service_utils.py
tests/server/services/storage/sqlite_storage/test_user_playbook_search_synthetic_request_visibility.py
tests/server/services/storage/test_lineage_b3_reconstruct_changelog_integration.py
tests/server/services/storage/test_lineage_b3_request_id_invariant_integration.py
tests/server/services/storage/test_storage_contract_playbook.py
tests/server/services/storage/test_storage_contract_profiles.py
tests/server/services/test_generation_service.py
tests/server/services/test_generation_service_durable_enqueue.py
tests/server/services/test_non_extraction_learning_metering.py
tests/server/services/test_prompt_model_mapping.py
tests/server/test_billing_meter_events.py

💤 Files with no reviewable changes (41)

tests/eval/reflection/init.py
tests/eval/reflection/fixtures/illustrative_cases.json
reflexio/server/prompt/prompt_bank/memory_reflection/v1.6.0.prompt.md
reflexio/server/services/reflection/init.py
reflexio/README.md
reflexio/server/prompt/prompt_bank/memory_reflection/v1.2.0.prompt.md
reflexio/server/prompt/prompt_bank/memory_reflection/v1.5.0.prompt.md
reflexio/server/services/reflection/components/init.py
tests/eval/reflection/fixtures/init.py
tests/server/services/reflection/test_reflection_service_utils.py
tests/models/test_reflection_config.py
reflexio/server/prompt/prompt_bank/memory_reflection/v1.4.0.prompt.md
reflexio/lib/_reflection.py
reflexio/server/prompt/prompt_bank/memory_reflection/v1.3.0.prompt.md
reflexio/server/prompt/prompt_bank/memory_reflection/v1.0.0.prompt.md
tests/eval/reflection/providers.py
reflexio/server/services/reflection/components/extractor.py
reflexio/server/prompt/prompt_bank/memory_reflection/v1.1.0.prompt.md
tests/server/services/reflection/test_module_contract.py
tests/eval/reflection/judge.py
reflexio/server/services/reflection/service.py
reflexio/server/prompt/prompt_bank/memory_reflection/v1.7.0.prompt.md
reflexio/server/services/reflection/reflection_service_utils.py
tests/server/services/reflection/test_post_horizon_filter.py
tests/eval/reflection/case.py
tests/server/services/reflection/test_generation_service_wiring.py
reflexio/server/services/README.md
reflexio/lib/reflexio_lib.py
tests/eval/reflection/runner.py
tests/server/services/test_prompt_model_mapping.py
tests/e2e_tests/test_failure_path_polarity_e2e.py
tests/server/services/reflection/test_reflection_profile_supersede_integration.py
reflexio/test_support/llm_model_registry.py
tests/server/services/test_non_extraction_learning_metering.py
tests/server/services/durable_learning/test_worker.py
tests/eval/scenarios/fixtures/scenarios.json
tests/server/services/reflection/test_reflection_service.py
tests/eval/reflection/test_reflection_eval.py
tests/server/llm/test_litellm_client_unit.py
reflexio/models/config_schema.py
tests/eval/reflection/README.md

📝 Walkthrough

Walkthrough

Reflection functionality is removed from the public facade, configuration schema, generation and durable-learning pipelines, service modules, prompt registry, and evaluation harnesses. Persisted legacy configuration cleanup, consolidation-only scenarios, lineage semantics, billing attribution, and related tests and documentation are updated.

Changes

Reflection subsystem removal

Layer / File(s)	Summary
Public contracts and configuration cleanup `reflexio/lib/`, `reflexio/models/`, `reflexio/server/services/configurator/*`	The `Reflexio` facade and `Config` schema no longer expose reflection, and legacy `reflection_config` keys are removed from local files when possible.
Runtime learning pipeline removal `reflexio/server/services/generation_service.py`, `reflexio/server/services/deferred_learning_plan.py`, `reflexio/server/services/reflection/*`, `reflexio/test_support/llm_model_registry.py`	Durable and classic learning no longer execute reflection stages, plans, persistence, side effects, or registry validation; the reflection service is deleted.
Consolidation-only evaluation flow `tests/eval/*`, `tests/models/test_removed_reflection_config.py`	Scenario and prompt-deviation evaluation paths now cover extraction and consolidation only, with reflection schemas, providers, fixtures, and tests removed.
Supporting tests, attribution, and documentation `reflexio/README.md`, `reflexio/server/README.md`, `tests/server/*`	Documentation, billing sources, lineage expectations, durable-learning tests, storage contracts, and playbook operation identifiers are updated for the remaining paths.

Estimated code review effort: 4 (Complex) | ~45 minutes

Possibly related PRs

ReflexioAI/reflexio#39: Introduced the reflection stack removed by this change.
ReflexioAI/reflexio#330: Related durable-learning compute/persist split changes involving the reflection plan.
ReflexioAI/reflexio#357: Modified the reflection pipeline that this change removes.

Suggested reviewers: yilu331, guangyu-reflexio

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 49.28% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main change: removing the unshipped reflection service.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch codex/remove-reflection-service

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

refactor: remove unshipped reflection service

3eba9d2

Delete the public API, config model, orchestration, prompts, evals, and service-only tests. Preserve legacy private config loading by stripping the retired raw key before strict validation.

yyiilluu merged commit 17ba810 into main Jul 18, 2026
1 check passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

refactor: remove unshipped reflection service - #360

refactor: remove unshipped reflection service#360
yyiilluu merged 1 commit into
mainfrom
codex/remove-reflection-service

yyiilluu commented Jul 18, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented Jul 18, 2026 •

edited

Loading

Walkthrough

Changes

❌ Failed checks (1 warning)

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

yyiilluu commented Jul 18, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Runtime and public API

Prompts, evals, and documentation

Test Plan

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Jul 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

❌ Failed checks (1 warning)

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

yyiilluu commented Jul 18, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Jul 18, 2026 •

edited

Loading